Neural signatures associated with temporal compression in the verbal retelling of past events

When we retell our past experiences, we aim to reproduce some version of the original events; this reproduced version is often temporally compressed relative to the original. However, it is currently unclear how this compression manifests in brain activity. One possibility is that a compressed retrieved memory manifests as a neural pattern which is more dissimilar to the original, relative to a more detailed or vivid memory. However, we argue that measuring raw dissimilarity alone is insufficient, as it confuses a variety of interesting and uninteresting changes. To address this problem, we examine brain pattern changes that are consistent across people. We show that temporal compression in individuals’ retelling of past events predicts systematic encoding-to-recall transformations in several higher associative regions. These findings elucidate how neural representations are not simply reactivated, but can also be transformed due to temporal compression during a universal form of human memory expression: verbal retelling.


Supplementary Figure 1. Cosine similarity between text embeddings derived from movie annotations versus spoken recall, separated by recall type for each subject.
Movie-to-recall text cosine similarity was separately computed for summary versus precise recall utterances. Each dot shows similarity values for an individual subject with lines connecting each subject's values for each recall type.

Supplementary Note 1. Testing for within-subject reinstatement effects in anterior hippocampus and posterior hippocampus
In addition to the parcel-level results reported in the main paper, we also tested for effects of 1) within-subject memory reactivation and 2) between-subject memory transformation in two anatomically-defined hippocampal ROIs: posterior hippocampus and anterior hippocampus.
For the within-subjects reactivation analysis at the micro-segment level, neither hippocampal ROI showed reliable reactivation effects during either summarized or temporally precise recall. In the posterior hippocampus, there was no reliable reactivation for microsegments that are later recalled in a temporally precise manner, t(16)= 1.5, p= 0.15, for microsegments that are later summarized, t(16)= 1.10, p= 0.28. This was also the case for anterior hippocampus: there was no reliable reactivation for micro-segments with temporally precise recall, t(16)= 1.6, p= .13, or for summarized micro-segments, t(16)= 0.48, p= .42.

Supplementary Note 2. List of stop words excluded from word counts of transcribed recall speech.
The following terms were considered stop words and thus were omitted from count of the number of words for utterances that contained them: a, an, and, are, as, at, be, by, for, from, has, in, is, its, of, on, that, the, to, was, were, will, with, uh, um, uhm, umm, uhh

Supplementary Note 3. Proportion of parcels from each functional network in the memory transformation mask
The proportion of tested parcels (i.e., those in the memory transformation mask in Figure  3a of the main text) where memory transformation scaled with summarization varied across networks: 44% of tested parcels in the Ventral Attention Network, 38% of the Fronto-Temporal parcels; 31% of the Visual Network Parcels; 28% of the DMN parcels; 22% of the Somato-Motor Parcels, and 20% of the Dorsal Attention Parcels.

Supplementary Figure 2. Average memory transformation values (recall-to-recall minus movie-to-recall) from the searchlight analysis reported in Chen et al. 2017.
Effects are shown in parcels where either recall-to-recall or movie-to-recall (between subjects) was reliable. For each reliable parcel (shown in Figure 3a, colored and light gray parcels), the memory transformation values observed in the parcel's searchlight centers were averaged together.

Supplementary Figure 3. Measures of split-half test-retest reliability for between-subjects pattern similarity across parcels.
There is no previous work to evaluate the test-retest reliability of our fMRI measurement of memory transformation, i.e., where recall-recall correlations (similarity of brain activity patterns across subjects during recall) exceed movierecall correlations (similarity of movie brain activity patterns with recall brain activity patterns across subjects). To compute test-retest reliability, we split the data in half (two sub-groups with random assignment, n=9 and n=8) and computed 1) movie-recall similarity 2) recall-recall similarity and 3) the difference between 1 and 2 (i.e., memory transformation) in each parcel. For more details on how these measures are computed, see pg. 26. We excluded parcels that did not show reliable effects for either 1) or 2) in the whole-group data, limiting analyses to the colored and light gray parcels depicted in Figure 3a (174/400 parcels). Thus, for each of the two subgroups, we computed the three measures in each of the 174 parcels. We then computed the Pearson correlation (r-value) between the 174 memory transformation values across the two subgroups, which quantifies the extent to which parcel-level memory transformation values are similar across the two sub-groups. We repeated this analysis 100 times with random splits of the 17 subjects into two sub-groups. The histogram below (top row) shows the distribution of rvalues (correlation of 174 memory transformation values across the two random sub-groups). The distribution average was r=0.14. Reliability (the distribution average) for recall-to-recall (measure #1, middle row) was r=.21 and reliability for movie-to-recall was r=.44 (measure #2, bottom row). In sum, for all three measures, we found consistently positive correlations between the parcel values for random split-halves of the data. Note that in order to avoid performing inter-subject pattern similarity analyses with too little data, scenes were excluded if they were recalled by less than five participants in each split-half group, leading to the inclusion of an average of 13.5/50 scenes across each the 100 random samples.

Supplementary Figure 4. Parcel-level map of the relationship between memory transformation and scene-level temporal compression factor (TCF) scores.
This relationship was only tested in parcels that showed reliable memory transformation effects (shown as light gray and colored parcels, see Figure 3a of main text). The parcel values yielded by this analysis are moderately correlated (r=.65) with the parcel values reported in the main manuscript text (see Figure 4b).